Backward Error Estimation

Estimation of unknowns in the presence of noise and uncertainty is an active area of study, because no method handles all cases well, or even satisfactorily. The basic problem considered is the linear system, Ax ≈ b, where A and b are given matrices with noise and uncertainty from measurements or modeling. The goal is to get the ”best” estimate of x. The problem is useful in a wide variety of situations since it covers how to invert a matrix that contains uncertainty. Mainstream methods like least squares and total least squares fail dramatically when A is ill-conditioned. Other methods like Tikhonov regression, ridge regression, and min max (bounded data uncertainty) provide robustness at the cost of fine details (by reducing ‖x‖). This paper presents a new family of regression methods based off the backward error criteria which can add robustness and when possible enhances fine details (by increasing ‖x‖). This paper covers the motivation, proof, and application of backward error estimation. The resulting algorithms are compared against existing methods in numerical examples from image processing. This paper is concerned with the estimation of unknowns that are related to some measurements by a linear model that is subject to uncertainty. Consider the set of linear equations, Ax = b, where A ∈ R and b ∈ R are given. The goal is to calculate the value of x ∈ R. If the equation is exact and A is not singular, the solution can be readily found by a variety of techniques, such as taking the QR factorization of A. Ax = b QRx = b Rx = Q b The last equation can be solved for x by back-substitution, since R is upper triangular. Given errors in modeling, estimation, and numeric representation the equality rarely holds. The least squares technique directly uses techniques like the QR factorization, by considering all the errors to be present in b. A more realistic appraisal of the system, considers errors in both A and b. Numerous methods exist for describing the errors in A and b, such as 1. bounding the norms of the errors in A and b, 2. constraining the errors in A and b to some structure, 3. partitioning A, b, and their corresponding errors, then placing bounds on the norms of each partition of the errors. Combinations of the methods to describe the errors are also considered by some techniques. The description of the errors and constraints is one of the two fundamental ways a technique is specified for the linear model Ax ≈ b. The other fundamental way of describing a method is to specify the cost function used to select the best value for x. Department of Electrical and Computer Engineering, University of California, Santa Barbara, CA 93106 ([email protected]). Computer Science Department, California State University, San Bernardino, CA 92407-2397 ([email protected]). Computer Science Department, California State University, San Bernardino, CA 92407-2397 ([email protected]). Computer Science Department, California State University, San Bernardino, CA 92407-2397 ([email protected]).